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Abstract 

We discuss a realization of the non-abelian group 0(2) as a family symmetry 
for the lepton sector. The reflection contained in 0(2) acts as a /x-r interchange 
symmetry, enforcing — at tree level — maximal atmospheric neutrino mixing and a 
vanishing mixing angle ^13. The small ratio m^/rriT- (muon over tau mass) gives rise 
to a suppression factor in the mass of one of the pseudoscalars of the model. We 
argue that such a light pseudoscalar does not violate any experimental constraint. 
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1 Introduction 



The discoveries of solar- and atmospheric-neutrino oscillations, besides having constituted 
remarkable experimental feats and having given neutrino theorists a much needed shot in 
the arm, brought with them the pleasant surprise that two of the lepton mixing angles 
seem to have (or are, at least, not far from) extreme values. Indeed, contrary to the 
solar-neutrino mixing angle, which has a large but non-maximal value, the atmospheric- 
neutrino mixing angle could be maximal (vr/4) and the third mixing angle, ^13, might 
vanish. These two features are easily explained, theoretically, by assuming the (effective) 
light-neutrino Majorana mass matrix Aiu, in the weak basis where the charged-lepton 
mass matrix is diagonal, to be /i-r symmetric [H [21 [3] : 



( X y y \ 

y z w 
\y w z J 



The mass matrix ([T]) is in very good agreement with the presently known data [1] . Let us 
write the /x-r interchange symmetry as 

S : Df,L ^ DrL, fJ-R ^ Tr, I^^r ^ V^R, 01 ^ 02, (2) 

where the (a = e, /x, r) are the left-handed-lepton gauge-S'f/(2) doublets, the VaR 
are right-handed-neutrino SU{2) singlets, which we add to the theory in order to enable 
a seesaw mechanism [5j, and the (pj {j = 0,1,2) are three Higgs doublets. This /i-r 
interchange symmetry s allows one to relate the small ratio of muon mass over tau mass, 
m^/mr, to a small ratio of vacuum expectation values (VEVs)|3 Indeed, if there is in the 
theory some extra family symmetry — besides s — such that only (pi has Yukawa couplings 
to the muon family and only 02 has Yukawa couplings to the tau family, then 

£y = ?/4 {D^L<Pl^-R + Dtl4>2Tr) + H.C., (3) 

hence m^/m,- = |t'i/f2|, where Vj j \pl = (o|0°|o) is the vacuum expectation value 
(VEV) of the neutral component of (pj. This may allow one to relate a property of the 
charged-lepton spectrum to features of the scalar potential and spectrum. 

In this paper we present a model in which the small ratio m^/m,- is related to a 
suppression factor in the mass of one of the pseudoscalars. One thus has an indirect 
connection between neutrino mixing properties and features of the scalar sector. Our 
model is particularly simple in that it uses the non-abelian group 0(2) as its main family 
symmetry. It has a scalar potential with less parameters than previous models, predicting 
in particular no CP violation. 

In section [2] we present the symmetries and the Lagrangian of our model. In section [3] 
we study the mass matrices of the scalars. Section H] is devoted to experimental con- 
straints on our model. We summarize our findings in section [5l Three appendices contain 
material which may be omitted in a first reading of our paper. Appendix A makes an 



similar mechanism has previously been used, for instance, in [6 . There, the ratio between the up- 
and charm-quark masses is equal to a ratio of two VEVs, in a model with horizontal symmetry S'3 x Z3. 
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abstract description of the group 0(2). Appendix B compares the present model with 
a previous model of maximal atmospheric-neutrino mixing with a naturally suppressed 
ratio m^/rrir [3 E]. Appendix C presents a variation of our model in which the symmetry 
s is substituted by a non-standard CP symmetry [9], with the practical consequence that 
one predicts "maximal CP violation" in lepton mixing instead of a vanishing mixing angle 

6*13- 



2 The model 

We consider an extension of the standard electroweak model (SM) with gauge group 
SU{2) X f/(l), with multiplets as described in the introduction: left-handed SU(2) dou- 
blets -Dql, right-handed SU{2) singlets and z/^j? (a = e, /x, r), and three Higgs doublets 
<P, (j = 0,1,2). 

The family symmetries of our model are the reflection symmetry s in ([2]) and also a 
f/(l) symmetry acting on the multiplets as 



U{1): 



e+*^ {D^L, tr, u^r) 



(4) 



Moreover, we need an extra symmetry (beyond s) given by 

^2 : VeR, i^f,R, i^rR, eR, 00 change sign. 



(5) 



The symmetry U{1) in (jl]) does not commute with the symmetry s in ([2]). One can 
conceive U{1) and s as generating together the non-abehan group 0(2), as discussed in 
appendix A. That appendix also contains the irreducible representations of 0(2). Equa- 
tions ([2]) and (jl]) may be interpreted in terms of those irreducible representation by the 
following assignments 



1 

2(1) 
2(2) 



{D^L, Drl) , {tr, Hr) 
(01, 02) ■ 



^iRi 



^tr) ; 



(6) 



The full family symmetry of the model is thus C = 0(2) x Z2. 

The above multiplets and symmetries determine the Yukawa Lagrangian 



Y 



-yi DeL4>0iyeR - y2 \ D^iL<Pol^^iR + DrL<Pol^TR^ 
-1/3 DeL4>0eR - Vi (-D^L0l/ii? + -DrL02T-i?) + H.C., 



(7) 



where 4>j = iT2(f)*. Because of the Z2 symmetry of ([5]) only 0o couples to the UaR and to 
cr. Because of the U{1) symmetry of (jl]) the Yukawa-coupling matrices are all diagonal. 
Due to the /x-r interchange symmetry of the neutrino Dirac mass matrix is given by 



M^) = diag (a, 6, b) 



(8) 



3 



with a = ylvo j and h = y^VQ f \pl. The charged-lepton masses are 



There is one VEV per charged-lepton mass. The mass ratio 

(10) 



V2 



is determined solely by a ratio of VEVs, the Yukawa couplings being totally absent there- 
from. 

An important ingredient of the model is the soft breaking of the U{1) of (jlj) — but 
neither of s nor of Z2 — by terms in the Lagrangian of dimension three or smaller. The 
family symmetry group 0(2) is softly broken to s: 

0(2) X ^ lli'^ X Z2, (11) 

where Zj*'' is the group generated by s. Later, Za'''' x Z2 is spontaneously broken by the 
VEVs of the Higgs doublets. The soft breaking ffTTj) permits the right-handed neutrino 
singlets to acquire Majorana mass terms, 

Cm = \vIC-^M*^vr + YL.c., (12) 

satisfying {Mr)^^ = {Mr)^^ and {Mr)^^ = (Mr)^^ because of s. Together, equations 
and f|T2|) determine the form of the effective Majorana mass matrix of the light neutrinos, 
Aiu = —MJ^M^^Md, to be as in equation ([1]). As stated in section [H this form of 
Ail, leads to two of the three lepton mixing angles having extreme values: ^23 = 7r/4 and 
^13 = 0, while the remaining mixing angle 612, and also the neutrino masses and Majorana 
phases, remain undetermined. 

3 The scalar sector 

3.1 The scalar potential and its minimum 

The scalar potential, taking into account the symmetries of the model, is given by 

V = fXo 0000 + yUl2 + 0202^ + /im ((plh + 0201 

+ai (^0j)0o) + (32 0000 (0l01 + 0202) + ^3 (0^)01 0l0O + 0^)02 020C 



+04 0001 0002 + al 0l0O 0200 + 



0l0l)'+(050 



2 

2 



+06 0^010202 + (37 0102 0201- (13) 

Because of the soft breaking ( fTTI) we have added to the potential a term 0^02 + 020i, 
which breaks the U{1) of (jll) but not the s of ([2]). In equation (|T3l) we are implicitly 
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assuming that there are in the model no scalar multiplets beyond 0o,i,2- The quarks have 
Yukawa couplings to 0o but neither to (pi nor to 02 — this situation may be enforced by 
suitably extending the symmetry of ([5]) to the quark sector. 

All the parameters in equation f|T3|) are real, except 04 which is in general complex. 
There are in the potential only two terms, the 04 term and the term, which can feel 
the two relative phases among the VEVs of the three doublets. Therefore, one can adjust 
those phases such that, simultaneously, the VEVs fo,i,2 are real and positive while both 
/Xm and a4 are real and negative. This arrangement minimizes V. Thus, from now on we 
shall use 

l^m = - |/im| , 04 = - WaI , Vo > 0, Vl > 0, V2 > 0. (14) 

The VEVs must fulfill two conditions: 

V ~ 246 GeV, (15) 

(16) 



V2 m-r 



where 



v = Jvi + vl + vl (17) 



The condition (]T5l) follows from the assumption that there are in the model no scalar 
multiplets beyond 0o,i,2- 

Writing (0 \V\ 0) as a function of Vq, vf + v^, and fif2, and enforcing the stability of 
(0 \ V\ 0) relative to each of these parameters, one obtains, respectively, 

/xo = -aivl-^[vl + vl) + \ai\viV2, (18) 

/ii2 = -§ ^0 - «5 [vl + vl) , (19) 

I I Qi^ 9 / r\r\\ 

|/im| = 77^1^2-— ^^0) (20) 
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where 



A = ag + At — 2a5, (21) 
B = as + 03- (22) 

These equations allow one to replace in a systematic way the parameters /iq, /ii2, and 
f^m by the VEVs. This replacement is convenient in order to calculate the masses of the 
scalars of the model in terms of independent parameters. Notice that it follows from 
equation ([20]) that A> 0. 

3.2 The mass matrices of the scalars 

We parameterize the Higgs doublets as 



{vj + pj + ir]j) / y/2 



(23) 
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with real fields pj and rjj. Since with our convention there are neither complex couplings 
in V nor complex VEVs, CP is conserved in the scalar sector, the fields pj are scalars 
while the rij are pseudoscalars, and there is no scalar-pseudoscalar mixing. The mass 
terms of the scalars are given by 



■'scalar masses 

) 

Pi 

After some algebra we find that the scalar mass matrices are 



"2 (Po, Pi, P2)M 



- t; iVo: Vi: m)Ml 




Ml = ^ 



as 
2 



VqVi 
VqV2 

/O 



2a5 - ae 



VqVi 







Ml 



Ml = 



-vl 
\ V1V2 



V0V2 \ 
] 
-4 I 
\ 

,2 



+ 



1^41 



-V0V2 -VqVi 







Vn 



-vt ) 

{Bvi — 1 04 1 V2) 2vo {Bv2 



04 Vl 



2vq {Bvi — |a4| -^2) 4a5f^ + Av^ 
\ 2vo {Bv2 - \ai\ Vl) (4a5 + A) V1V2 



(4a5 + A) V1V2 
Aa^vl + Avl 



I 





^ '2,ViV2 


-VqV2 


-VqVi 


\ 


A 


/o 










-VqV2 









+ 2 





vl 


-VIV2 




\ -VqVi 







) 




lo 


-V1V2 


vl 



Both and M"^ have an eigenvector 




(24) 



(25) 



(26) 



(27) 



(28) 



with eigenvalue zero; the corresponding scalar fields are the unphysical scalars (Goldstone 
bosons) and (7°, associated with the and gauge bosons, respectively. We denote 
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the diagonalization of A^^, A^^, and A4'^ by 



respectively. Thus, 



\vt ) 

Pi 
J 

Vi 

\V2 J 




1,2, 



Mix, 
Mix, 



plXb 



for a 
for b 
for 6 = 4,5. 



1,2,3, 



(29) 



(30) 



(31) 



(32) 
(33) 
(34) 



The ml (a = 1, 2) are the squared masses of the charged scalars, the /i^ (6 = 1,2, 3) are 
the squared masses of the neutral scalars, and the f^l {h = 4, 5) are the squared masses of 
the neutral pseudoscalars. The decomposition of the Higgs doublets in physical fields is 
given by 



2-1/2 



Ell (Xt), S', + I (Xo)^. GO + z EL4 (X,)^ S\ 



(35) 



3.3 The light pseudoscalar 

The non-zero eigenvalues of the mass matrix of the pseudoscalars are determined by 

a = + pI= {vl + vl) (^ + k\ai\^ (36) 



and 

where we have defined 



p = pIpI 



2ViV2 



2m^mr 



mj, + ml 8.44 



(37) 
(38) 



Fixing 5*4 to be the lightest one of the two physical pseudoscalars, i.e. 



pI 
pI 



a 



4p 



(39) 



we see in equation ( !37|) that = if either |a4| = or |a4| = AviV2/vq; the latter case 
means |/im| = — see equation (l20l) . This is easy to understand: 
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• If /^m = 0, then the U{1) of (jl]) is unbroken in the scalar potential. This implies the 
existence of one physical neutral Goldstone boson, corresponding to an extra [i.e. 
beyond Xq) eigenvector (0, Vi, —V2) of with eigenvalue 0. 

• If 04 = 0, then there is an additional f/(l), 0o ~^ e*^0O) unbroken in the scalar 
potential. This implies the existence of one physical neutral Goldstone boson, cor- 
responding to an extra eigenvector (1, 0, 0) of ^A'^ with eigenvalue 0. 

• If both /iin and 04 vanish, then there are three U{1) symmetries, (^j — >■ e*"-'0j for 
j = 1,2,3, unbroken in the scalar potential. This might be thought to imply the 
existence of two physical neutral Goldstone bosons. However, when both /im and 04 
vanish, vi also vanishes jl which means that one of those three U{1) symmetries is 
not spontaneously broken. Therefore, in that situation there is again one physical 
neutral Goldstone boson, corresponding to an extra eigenvector (^2, 0, —Vq) of Ai"^ 
with eigenvalue 0. 

Since we know Vi to be much smaller than V2, we expect, in general, both Ifx^^l and 
1 04 1 to be relatively small, and therefore we expect to be relatively light. In order to 
quantify and qualify this expectation we consider 



where x = a / ^/p. Since 

dx 



4 = (40) 

yu.4 X — va; — 4 



> 0, (41) 



{x - y/x^ - 4 ) Vx^ - 4 



/igZ/il decreases when x decreases. Equations (!36l) and (j37l) determine a: as a function of 
1 04]. Minimizing x with respect to |a4|, one finds 



r , Vn 



2Wl-r + — , where r = -|. (42) 



k"^ V 



Inserting Xmm into equation (140|) . one obtains 

fil \Jr — k'^r + /c^ + — k^r ^^^^ 

/^l jnin y/r — k'^r + k"^ — \Jr — k'^r 

This lower bound on fx^/ is depicted in figure [H It is seen that, unless Vq is very smalljf] 
the lighter pseudoscalar will in general be ten or more times lighter than the heavier 
pseudoscalar. But, Vq cannot be too small, lest the Yukawa coupling responsible for the 
top-quark mass needs to be very large. 



^Equation ([20|) holds trivially in this case. 
•^Note that fi5/^4\^-^^^ = 1 for vq = 0. 
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Figure 1: The minimum possible value of /i5//i4 as a function of v^/v. 
3.4 The eigenvectors in the hmit vi = {) 

As a preparation for the next section, we now investigate the limit f i = in detail. In 
that limit 04 and /Xm must also vanish, hence the scalar mass matrices are given by 



Ml 



Mt 



f -a-svl 











a3VoV2 ^ 







+ (ae - 205) vl 





\ a3VoV2 













( ^aivl 







2BVqV2 ^ 









Avl 
















Aa^vl ) 






/ 


o\ 










Avl 













\ 


J 











(44) 



(45) 



(46) 



From the positivity of the mass matrices Ail, and TVI^ we then have 



03 < and 4aia5 > (02 + 03)^ , 



(47) 
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respectively. Notice that ai and 05 must be positive, even when vi 7^ 0, in order for the 
potential to have a minimum. 

Equations fH^ - fH6|) show that 0i completely decouples from 0o and 02 in the limit 
f 1 = 0. In that limit one has 



mo 



03 2 , /Q-e 



/'^6 \2 2 2^2 2n-l/'\^ \^ 

VY ~ ^2 = = Y ^2, = 0, ^2 = ^2 = ^5 

Moreover, equation readily gives 



1 



2 '^3 2 1 

= V , Yi = - 



[ -V2\ 



V ^0 / 



(45 



(49) 



while equation fH5]) leads to 



/i^ 3 = aivl + a^vl ± y {aivl - a^v^)^ + B'^VqV. 



2.,2 

25 



and 





/ 


cos A 


\ 




/ 


sin A \ 




















V 


— sin A 


J 




v 


cos A J 



, tan2A 



BvqV2 



(50) 



(51) 



4 Phenomenology of the scalar sector 

The aim of this section is to demonstrate that the model presented in this paper complies 
with all the experimental constraints. We remind the reader that the masses of the 
physical scalars, and the mixing angles contained in the eigenvectors (a = 1, 2) and 
Xb {h = 1, . . . , 5), are functions of the scalar-potential quartic couplings oi, . . . , 07 and of 
Vo/v. It is beyond the scope of this paper to perform a complete exploration of this large 
parameter space; we shall restrict ourselves to show that it is possible to find a set of 
parameters such that the scalar sector of the model does not contradict any experimental 
results. We will choose one such set and call it 'the reference scenario'. 

Although the present model is mainly designed for the lepton sector, one may extend 
it to the quark sector, as mentioned in section 13.11 The obvious way to do this is to 
stipulate that under the Z2 symmetry in ([5]) the right-handed-quark gauge-S'f/ (2) singlets 
transform with a minus sign. Then, only 0o has Yukawa couplings to the quark sector. 
In this way there are, just as in the SM, no flavour-changing neutral Yukawa interactions 
of the quarks. This extension resembles in its spirit a type-I two-Higgs-doublet model 
(2IIDM). We require Vq ^ 100 GeV in order to avoid a top-quark Yukawa couphng much 
larger than unity. 

The Lagrangian for a generic multi-Higgs-doublet model (MHDM) can be found in [10] . 
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4.1 Constraints from decay 

Z° decay into charged scalars In a MHDM, the Z° couples to S^S~ with a universal 
strength, independent of the details of charged- scalar mixing; the relevant term in the 
Lagrangian is 



2c 



w 



A model-independent lower bound on the masses of the charged scalars can be derived 
from the invisible decay width of the Subtracting from it the SM decay width of the 
into neutrinos, the difference is compatible with zero, leaving little room for an additional 
decay of the Z° into charged scalars [11]. This results in the bound [12] 



nia > 43.7 GeV (95% CL) , a= 1, 2. (53) 



Higgs strahlung From LEP data, a lower mass limit nih > 114.4 GeV has been de- 
duced \13\ for the SM Higgs particle h, from the unobserved "Higgs strahlung" process 
e~^e~ Z* ^ Zh. Note that this process is allowed only for scalars but not for pseu- 
doscalars [H]. In the present model, all three scalars 5° 2 3 can in principle be produced 
by Higgs strahlung; the relevant term in the Lagrangian is 



9mz 



,=1 



Z^Z^5^(Xo-X,)5°, (54) 



where the quantity in parentheses denotes the scalar products of the vectors Xq and Xb. 
In the limit Vi = the production of 5'° is suppressed since Xq ■ X2 = 0, as can be read 
off from equations fl28l) and fHHj) . On the other hand, in that limit the strengths of the 
couplings of Si and ^3 are complementary, with [Xq ■ Xi^ + (Xq ■ Xs)^ = 1. 

Associated production The Z^ can decay into a scalar-pseudoscalar pair [T4]; the 
relevant term in the Lagrangian is 

3 5 

E E ■ ^^') i^'d^^'b' - S'b'd^S',) . (55) 



2cw 

^ b=l b'=A 



The lightest pseudoscalar of our model, S^, can in general be produced in this way asso- 
ciated with either or 5*3, but not with 5*2, since X4 ■ X2 = in the limit of vanishing 

Vi. 



4.2 Constraints from other decays 

Decays of the charged scalars The 2HDM has a single charged scalar H^. Assuming 
BR{H+ r+z/^) + BR(i/+ ^ cs) ~ 1, the bound mH+ > 78.6 GeV (95% CL) has been 
derived from the combined LEP data [13]. One cannot use this bound uncritically in the 
present model, which has two charged scalars and in which BR(S'^ —>■ ^'^Vp) is certainly 
non- negligible. Still, the bound on mH+ suggests an estimate of how much the bound fl53l) 
can possibly be raised by taking into account specific decay channels of S*^. 
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vq/v ai 02 as 


04 


/ 


^4 Lax '^5 cte aj 


I/V2 2.5 3 -5 




0.4 1.5 2 3 



Table 1: Input values for the reference scenario. 

Other decays The transition 6 — 57 is important because it provides an indirect, yet 
quite stringent, lower bound on the charged- scalar masses [IS] Since only 0o has Yukawa 
couplings to the quarks and the S2 component of ip^ is suppressed, the lower bound from 
b S'j applies only to mi. Vector mesons could possibly decay into a very light scalar 
plus a photon [18], yielding a lower bound on the scalar mass. This is relevant for the 
decay T (IS*) — > 6*47. Loop corrections in the decay bb are also important [I9] in 

the 2HDM for large tan/5. However, since our model has features similar to a 2HDM with 
tan/3 ~ 1, in which range this decay is not stringent [16\, we will disconsider it in the 
following. 

4.3 "Safe" scalar masses 

In the light of the above discussion we require 

mi > 350 GeV, /ii > 120 GeV, fi3 > 120 GeV, /i4 > 10 GeV. (56) 

Some remarks are at order. In the 2HDM of type II the bound on the charged-scalar 
mass from 6 — 57 is of the order of several hundred GeV, much larger than the bound 
from direct LEP searches. We have rather arbitrarily set that bound to 350 GeV in (l56ll . 
by considering the results obtained in [15] for tan/? ~ 1 and taking into account the 
considerable uncertainty in the computation of the corresponding 5-meson decay. The 
bounds on /ii and /is have been stipulated in order to definitely avoid production via Higgs 
strahlung. Finally, the lower bound on /i4 stems from the wish to avoid any problems from 
T(IS') S^'j. We have not put lower bounds on m2, /i2, and /is in fl56l) because, from 
the discussions in the previous paragraphs, we conclude that there are no really stringent 
bounds on these masses. Of course, these masses should not be too small. In any case, 
numerically it will turn out that if we fulfill the constraints of fISB]) . then also m2 and fi2 
will be reasonably large. Moreover, we bear in mind that in our model /is S> /i4 holds 
anyway. 

4.4 A reference scenario 

In table [1] we have written down a set of values for the eight parameters of the model, 
which we define to be our 'reference scenario'. All input values are of order one, except as 
which is somewhat larger because it is responsible for a large mass mi — see equation (jUj). 
In table m |a4|max = AviV2/vq is the maximal value of |a4|, obtained from equation fl2Ul) . 

*See also [16l[T7] and the references therein. 



12 



Taking the input from table [T] and performing a numerical calculation, we obtain 



mi = 389.2 GeV, ms = 245.8 GeV, 

/ii = 434.8 GeV, /i2 = 171.9 GeV, /xg = 231.8 GeV, (57) 
/i4 = 14.3 GeV, /is = 173.8 GeV. 

These values agree well with the ones computed from the approximate formulae of sec- 
tion [331 For instance, fi2 — /^s in fl57|l . The masses (1571) satisfy the conditions fl56l) for 
"safe" masses. 

Next we check the reference scenario against electroweak precision data by using the 
oblique parameters pO] S, T, and U. For a MHDM we take the formula for T in (for 
computations of T in the 2HDM, see e.g. [TU [22l [23]), which gives, when applied to the 
present model 

1 



J]J](F..X,)^FK,/.^) 

a=l 6=1 

-J]5^(X,.X,)^F(/.^, 

b=l b'=4 
3 

+3 ^ (Xo ■ Xbf [F (ml, fil) - F (m^^, /i^] 



levrs^m^ 



6=1 



-3[F(m|,m2)-F(m2^, m^)] i, (5^ 



where 



F(x,y) = ^-^ln^. (59) 
2 x-y y 

In equation (1551) . mw and m^ are the masses of the and gauge bosons, respectively, 
= 1 — m^/m|, and rrih is the mass of the SM Higgs boson. One may also write down 



formulae for 5* and for U by applying the results in [21] . Taking the central value rrih = 87 
GeV from recent SM fits j2l] and using the scalar masses and diagonalizing matrices of 
the reference scenario, we have obtained S = 0.046, T = —0.162, and U = —0.002. These 
values are compatible with the fit results for the oblique parameters given in [25]. We 
note that, while all the individual contributions to S and to U are small and no excessive 
cancellations occur among them, this is not so for T: considering separately the first and 
the second terms in the right-hand side (RHS) of equation (158|1 . each of them is one order 
of magnitude larger than the final result T = —0.162; however, those two contributions 
have opposite signs (note that -F > 0), leading to a partial cancellation. Nevertheless, 
a certain amount of tuning of the input parameters is expedient to achieve the correct 
order of magnitude of T, as will be discussed in the next paragraph. The numerical value 
of third term of the RHS of equation fl58l) is naturally one order of magnitude smaller 
than the values of the first and second terms, and the SM subtraction in the forth term 
is numerically a tiny effect. 

In figure[2]we have attempted to illustrate the dependence of T on the input parameters 



13 



1 2 3 4 5 

«7 



Figure 2: The oblique parameter T as a function of ai. The input parameters not shown 
in the plot are as in table [TJ 



ai and 02; these occur only in the mass matrix of the neutral scalars (l26ll and might, 
therefore, be able to disturb the cancellation between the first and second terms in the 
RHS in equation fl58l) . In figure [2] we have plotted two curves. In the first one we have 
fixed 02 = 3 to its value in the reference scenario, whereas in the other one we have fixed 
fli + 02 = 5.5; the crossing point of the curves corresponds to the reference scenario. The 
figure illustrates nicely the tuning required for keeping T small. We note that the curve 
with fixed a2 begins at ai = 1, where /is = 115 GeV is outside the region of "safe" masses, 
but /is quickly grows with Oi. 



5 Conclusions 



In this paper we have presented a model for the lepton sector based on the family sym- 
metry 0(2). The model has an obvious extension to the quark sector, by coupling to the 
quarks only the Higgs doublet 0o which transforms trivially under 0(2). The smallness 
of the masses of the light neutrinos is explained in our model through the seesaw mech- 
anism. The reflection symmetry contained in 0(2) acts interchange symmetry]^ 
which — together with the U{1) C 0(2) — enforces diagonal Yukawa-coupling matrices and 
a neutrino mass matrix of the form ([1]). Consequently, the model predicts ^23 = 7r/4 and 
^13 = at the tree level. In that respect the model in this paper is practically identical to 

^In appendix C we show that it is possible to replace the reflection symmetry by a non-standard CP 
transformation; in that version of the model there is no 0(2) family symmetry. 
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the one in [7]; in both models the lepton flavour violation resides only in the mass matrix 
of the right-handed neutrinos. It has been shown ^26j that models of this class are safe 
from flavour- changing neutral interactions. 

The crucial difference between the present model and the one in [7j is that, in the 
0(2) model, we allow a non-trivial transformation of the Higgs doublets 0i and 02 under 
the U{1) C 0(2). In this way, we obtain a relation between the smallness of the mass 
ratio m^/mj. and the small mass /i4 of one of the two pseudoscalars of the model; indeed, 
that pseudoscalar is almost a Goldstone boson and only a soft U{1) breaking in the scalar 
potential prevents yU4 from being exactly zero. On the other hand, that soft breaking must 
necessarily be small in order to reproduce the small value of m^/mi- = fi/f2, determined 
by the ratio of the VEVs of 0i and 02!^ 

It had previously been realized [161 1221 that, in the 2IIDM, one of the neutral scalars 
could be quite light without contradicting any experimental constraints. We have at- 
tempted to show that the same holds in our three-Higgs-doublet model. Actually, our 
model not only predicts the light pseudoscalar, it also predicts the near equality of the 
mass of one of the scalars and the mass of the heaviest pseudoscalar, and, in addition, 
specific features in scalar mixing, resulting from the near decoupling of 0i from 0o and 
02, due to the smallness of m^/mr- We have thus demonstrated that, within our 0(2) 
model, the connection between the lepton and scalar sectors can be much tighter than 
usually thought of. 



A The group 0(2) 

Definition and characterization 0(2) is the group of rotations and reflections of the 
plane. It is generated by rotations g (9), with angle 9, around the center of the coordinate 
system, and by the reflection s about the x-axis. Allowing the angle 9 to vary over R, 
the properties of these group elements, which fully characterize the group, are 

g{9 + 2n)=g{9), g (9,) g (92) = g {9, + 9^) , = e, s g (9) s = g {-9) . (Al) 

Irreducible representations There are two singlet irreducible representations of 0(2): 

1:^(0)^1,5^1 and V : g (9) ^ 1, s ^ -1. (A2) 

Furthermore, 0(2) has a countably infinite set of doublet irreducible representations, 
numbered by G N: 

®It is interesting to note that, if both the U{1) and the reflection symmetry s are softly broken in the 
scalar potential, then vi = still implies = 0, 04 = 0, and a Goldstone boson. Thus, the prediction 
Hi <C 115 remains unaltered. 
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Tensor product 2^""^ ® 2^") We assume that the matrices in (1A3P act on an orthonormal 
basis {ci, 62}. In the product 2^™-* (8> 2*^"-' we must distinguish two cases. If m > n, then 

The irreducible representations in the right-hand side of flA4p have basis vectors 

2('"+") : ei ®ei, 62 62 and 2(™-") : 61 ® 62, 62 ® 61. (A5) 

If m = n, then 

The irreducible representations in the right-hand side of (1A6I) have basis vectors 

1: ^(61® 62 + 62® 61), 1': ^(61® 62 -62® 61), 2(2") : 61® 61, 62 062- (A7) 
V 2 V 2 

B Comparison of the present model with the model 
of softly broken lepton numbers 

The Z2 model The model presented in this paper — let us call it "0(2) model" — is quite 
similar to the model proposed by two of us a few years ago [7] — let us call it "Z2 model" . 
The Z2 model has the same fermion and scalar multiplets as the 0(2) model. Both the 
^2 and 0(2) models have the s of ([2]) and the Z2 of ([5]) as symmetries. However, instead 
of the U{1) of (jl]), employed as a symmetry in the 0(2) model, the Z2 model requires the 
conservation, in all terms of dimension four in the Lagrangian, of the three family lepton 
numbers. As a consequence, the Yukawa Lagrangian of the Z2 model has, beyond the 
terms in equation ([7j), one further term: 

Cy = {D^L(j)2lJ'R + D-,l(PiTr) + H.c. (Bl) 

Therefore, in the Z2 model the ratio between the muon and tau masses is 

nij. 

Symmetry group 0(2) in the Z2 model It was noted as a side remark in [28j that 
the Z2 model also has family symmetry 0(2). This group 0(2) is generated by the /i-r 
interchange symmetry s together with the U{1) of the lepton number L^ — L-r- Replacing 
01 and 02 by (j)± = {(pi ± ^2) /v^, we see that, under that 0(2), 0+ transforms as a 1 
and 0_ as a 1'. The 0(2) model, on the other hand, has two Higgs doublets transforming 
as a 2*^2'* of 0(2), instead of as a 1 © V; one further difference is that the f/(l) group in 
the 0(2) model is not really — Lr, cf. (jl]). 



2/4f 1 + y5V2 



1/4^2 + y^vi 



(B2) 
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Naturally small m^/rrir in the Z2 model In [8] an additional symmetry, dubbed K, 
was introduced into the model in order to provide a technically natural explanation 
for the smallness of nif^/mr- Under K, (pi and /ir change sign while all other fields remain 
invariant. The symmetry K eliminates the term — see equation fIBip — from the Yukawa 
Lagrangian of the model, thus obtaining m^/m,- = |fi/f2| just as in the 0(2) model. 
We want to stress that, from the point of view of neutrino masses and lepton mixing, the 
0(2) model of the present paper is equivalent to the Z2 model of [7] and also to the Z2 
model with the additional symmetry of [8]. The difference lies in the scalar potential, 
which in the 0(2) model is both different and more restricted. Indeed, in the Z2 model 
with a softly broken symmetry K, the 04 term is absent from the scalar potential; on the 
other hand, there are extra terms 




with bi real but 62 in general complex. The model of |8] has the advantage, over the 0(2) 
model, that m^/m-r is small in a technically natural sense; indeed, in that model f 1 7^ 
only obtains when K is softly broken by the /is term, while in the 0(2) model vi 7^ 0, 
even if = 0, because of the term. The advantage of the 0(2) model is its prediction 
of a light pseudoscalar — a prediction inexistent in the model of [8]. 



C Substitution of the symmetry s by a non-diagonal 
CP symmetry 

In the model suggested in this paper it is possible to use, instead of the /i-r interchange 
symmetry s, the non-trivial CP symmetry l29| 

02, 

and C is the Dirac-Pauli charge conjugation matrix. This CP symmetry commutes with 
both the U{1) of (jlj) and the Z2 of ([5]), so that, in this case, the model has symmetry 
CP X f/(l) X Z2 instead of 0(2) x Z2. Instead of equation (JTj) we would then have 



CP: < 



01 
02 





/ 1 





0\ 
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0/ 



(CI) 



c 



Y 
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X and w being real. Such a model predicts [0] maximal atmospheric-neutrino mixing 
(^23 = 7'"/4) but, instead of U^z = 0, it predicts [2l [9] \U^i\ = \Uri\ for all i = 1,2,3 {U is 
the lepton mixing matrix), which leads to sin ^13 cos 5 = 0, with 6 being the CP-violating 
phase in the mixing matrix. Although this condition permits ^13 = 0, it can be shown 
that the more general case is that of maximal CP violation [9] i.e. 6 = ±7r/2. The scalar 
potential is the same as in equation ( fT3i) with the proviso ( fT^ . 
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